Qwen3-Next-80B-A3B-Thinking is a new-generation thinking large language model launched by the Tongyi Qianwen team. It adopts an innovative hybrid attention mechanism and a highly sparse MoE architecture. While maintaining efficient reasoning, it has powerful complex reasoning capabilities and natively supports a context length of 262K.

Natural Language ProcessingTransformers

DeepSeek-V3.2-Exp-AWQ is a quantized version based on the DeepSeek-V3.2-Exp model, which achieves efficient text generation through the vLLM framework. This model introduces the DeepSeek sparse attention mechanism, significantly improving the efficiency of long context training and inference while maintaining the quality of the model output.

Natural Language ProcessingTransformers

Qwen3-Next-80B-A3B-Instruct is an efficient sparse mixture of experts model developed by the Tongyi Qianwen team. It has a total of 80 billion parameters, with only 3 billion active parameters. The model adopts an innovative hybrid attention mechanism and a MoE architecture with an extremely low activation rate, significantly improving inference efficiency while maintaining strong performance. It natively supports a context length of 262,000 tokens and can be extended to 1 million tokens.

Natural Language ProcessingTransformers

Qwen3-Next-80B-A3B-Instruct is the latest generation of large language model developed by the Tongyi Qianwen team at Alibaba. It adopts an innovative hybrid attention mechanism and a highly sparse mixture-of-experts architecture. While maintaining a total of 80B parameters, only 3B parameters are activated, achieving efficient context modeling and inference acceleration. It natively supports a context length of 262K and can be extended to 1M tokens.

Natural Language ProcessingTransformers

Qwen3-Next-80B-A3B-Thinking-AWQ-4bit is a 4-bit version generated from the Qwen3-Next-80B-A3B-Thinking model through the AWQ quantization technology, which significantly improves the model's inference efficiency. This model adopts a hybrid attention mechanism and a highly sparse MoE architecture, and performs excellently in complex reasoning tasks. It supports a context length of up to 262K tokens.

Natural Language ProcessingTransformers

Qwen3-Next-80B-A3B-Instruct-AWQ-4bit is an efficient version based on the Qwen3-Next-80B-A3B-Instruct model with 4-bit AWQ quantization. This model adopts a hybrid attention mechanism and a highly sparse mixture-of-experts (MoE) architecture, supporting a context length of up to 262K tokens. It significantly reduces the computational resource requirements while maintaining high performance.

Natural Language ProcessingTransformers

Qwen3-Next-80B-A3B-Instruct is the base model of the Qwen3-Next series. Through technologies such as the innovative hybrid attention mechanism and highly sparse mixture-of-experts layers, it achieves efficient modeling and reasoning acceleration in long text processing and demonstrates excellent performance in multiple benchmark tests.

Natural Language ProcessingTransformers

FastWan2.1-T2V-1.3B-Diffusers is a video generation model fine-tuned based on the new Sparse-distill strategy. It combines the advantages of distillation and sparse attention, supports efficient 3-step reasoning, and can generate high-quality videos at a resolution of 61×448×832.

MultimodalDiffusers

The Linglong Linear Preview is a hybrid linear sparse large language model open-sourced by InclusionAI, with a total of 17.1B parameters and 3.0B activated parameters. This model implements long-text reasoning based on a hybrid linear attention mechanism, achieving near-linear computational complexity and near-constant space complexity during inference.

Natural Language ProcessingSafetensorsMultiple Languages

Pino is a Dutch pre-trained model based on the BigBird architecture, utilizing sparse attention mechanisms to handle long sequence texts, supporting sequences up to 4096 in length.

Natural Language ProcessingJaxOther

The LSG model is an improved long-sequence processing model based on BART-large, utilizing local + sparse + global attention mechanisms for efficient handling of long-text tasks

Natural Language ProcessingTransformersEnglish

Korean Transformer model based on sparse attention, capable of processing sequences up to 4096 in length

Natural Language ProcessingTransformersKorean

A Transformer model based on sparse attention, fine-tuned specifically for mobile app description classification tasks

Natural Language ProcessingTransformersEnglish

BigBird is a Transformer model based on sparse attention, capable of processing sequences up to 4096 in length, suitable for tasks like long document summarization.

Natural Language ProcessingTransformersEnglish

BigBird is a Transformer model based on sparse attention, capable of handling longer sequences, suitable for tasks like long document summarization.

Natural Language ProcessingTransformersEnglish

BigBirdPegasus is a Transformer model based on sparse attention, capable of handling longer sequences, especially suitable for long document summarization tasks.

Natural Language ProcessingTransformersEnglish

BigBird is a Transformer model based on sparse attention, capable of processing sequences up to 4096 tokens long, suitable for long document tasks.

Natural Language ProcessingJaxEnglish